You can download the raw source code for these lecture notes here.

Course Meeting Plan

Wednesday · 12 February · Lecture

  • Course intro and demo portfolio (10 min)
  • Lecture: What is computational musicology? (15 min)
  • Breakout 1: Spotify features (15 min)
  • Plenary Discussion: Breakout findings (10 min)
  • Lecture: What is a spectrogram? (15 min)
  • Demo: Sonic Visualiser (10 min)
  • Lecture: What is good analytical design? (15 min)

If you want some distraction after the lecture, have a look at the Sources of Good Data Visualisations on Canvas.

Wednesday · 12 February · Lab

Please have your software set up following the Canvas directions before the lab session.

  • Welcome and introductions (15 min)
  • Demo: RStudio and R Markdown (15 min)
  • Breakout 2: AI Song Contest (20 min)
  • Plenary Discussion: Breakout findings (5 min)
  • Demo: ggplot (15 min)
  • Breakout 3: Plotting (20 min)
  • Plenary Discussion: Breakout findings (10 min)

Breakout 1: Spotify Features

Visit the Spotify feature documentation. It is quite short – and Spotify provides no other explanation of what their features mean.

For each feature, discuss with your group which of the four ‘musical moments’ you think Spotify uses to compute it:

  1. Pitch
  2. Volume
  3. Timbre
  4. Duration/Time

Some of them are obvious, and some of them are very much not!

Be prepared to discuss your conclusions with the class at the end of the breakout.

Breakout 2: AI Song Contest

In 2020, Dutch public broadcaster VPRO launched an AI Song Contest to promote the creative use of AI in music. The contest has recurred annually since then, most recently in Zürich, Switzerland. To get started learning about the possibilities of audio analysis, we will look at some core musical features generated by the Essentia library for last year’s AI Song Contest semi-finalists.

Introduction

Every analysis you do for this class must start with the following lines, which loads the tidyverse family of packages you learned about on DataCamp. The tidyverse ‘dialect’ of R is the dominant R style in data science today, and it is easier to learn than older styles of R coding.

library(tidyverse)
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr     1.1.4     ✔ readr     2.1.5
## ✔ forcats   1.0.0     ✔ stringr   1.5.1
## ✔ ggplot2   3.5.1     ✔ tibble    3.2.1
## ✔ lubridate 1.9.4     ✔ tidyr     1.3.1
## ✔ purrr     1.0.2     
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

Next, we will load the data from the AI Song Contest corpus. First, download the corpus features here and save them to your R project for the course. During the lab, we will show you how to load data using the graphical interface RStudio, which also will show you the command that you need to load the data automatically from within R Markdown.

aisc2024 <- read_csv("aisc2024.csv")
## Rows: 32 Columns: 7
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (2): artist, title
## dbl (5): danceability, valence, arousal, instrumentalness, tempo
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

For each of the AI Song Contest semi-finalists, these data include seven data points, or ‘features’ in AI terminology: - artist: the name of the artist - title: the title of the track - danceability: an estimate of how likely a song is to be ‘danceable’ (0 = very unlikely to 1 = very likely) - valence: an estimate of positive/happy the track sounds (1 = entirely negative/sad to 9 = entirely positive/happy) - arousal: an estimate of how energetic the track sounds (1 = very low energy to 9 = very high energy) - instrumentalness: an estimate of how likely the track is to be an instrumental track without vocals (0 = very unlikely to 1 = very likely) - tempo: an estimate of the overall tempo of the track (in beats per minute)

After you load these features, the easiest way to look at them is to click on the variable in the Environment tab in the top-right pane of RStudio. This will open a full, sortable, spreadsheet-style summary of your data.

You can also compute summary statistics with the summary() function.

aisc2024 |>
  summarise(
    mean_tempo = mean(tempo),
    sd_tempo = sd(tempo),
    median_tempo = median(tempo),
    mad_tampo = mad(tempo)
  )
## # A tibble: 1 × 4
##   mean_tempo sd_tempo median_tempo mad_tampo
##        <dbl>    <dbl>        <dbl>     <dbl>
## 1       113.     32.6          117      54.9

Instructions

Look at the data in the AI Song Contest corpus, starting in spreadsheet form, and sort by different features of interest.
Try identifying outliers on the playlist – tracks that have extreme and unusual values on one of the Essentia features – and listen to them. Do you agree with Essentia’s assessment?

Now compute summary statistics for some of the features that interest you. How accurately do you think these features capture your overall sense of the AI Song Contest music? Do some features seem more reliable than others?

Breakout 3: Digster Decade Comparisions

In this breakout session, you and your group will try to make the best visualisation you can to give an overall picture of the AI Song Contest 2024.

Introduction

The simplest data visualisation of a single, continuous variable is a histogram.

aisc2024 |> ggplot(aes(x = tempo)) + geom_histogram(binwidth = 10)

To compare two continuous variables, you can create a scatter plot. If you have a large corpus and the points end up on top of each other, geom_jitter() adds some random noise without disturbing the overall pattern.

aisc2024 |> ggplot(aes(x = tempo, y = arousal)) + geom_point() + geom_smooth()
## `geom_smooth()` using method = 'loess' and formula = 'y ~ x'

aisc2024 |> ggplot(aes(x = tempo, y = arousal)) + geom_jitter() + geom_smooth()
## `geom_smooth()` using method = 'loess' and formula = 'y ~ x'

Good visualisations, of course, have many more components. Remember Tufte’s six design principles (2006):

  1. Show comparisons, contrasts, differences.
  2. Show causality, mechanism, explanation, systematic structure.
  3. Show multivariate data; that is, show more than 1 or 2 variables.
  4. Completely integrate words, numbers, images, diagrams.
  5. Thoroughly describe the evidence. Provide a detailed title, indicate the authors and sponsors, document the data sources, show complete measurement scales, point out relevant issues.
  6. Analytical presentations ultimately stand or fall on the quality, relevance, and integrity of their content.
aisc2024 |>                   # Start with the data
  ggplot(                     # Set up the plot.
    aes(
      x = tempo,
      y = arousal,
      size = instrumentalness,
      colour = danceability
    )
  ) +
  geom_point() +              # Scatter plot.
  geom_rug(linewidth = 0.1) + # Add 'fringes' to show data distribution.
  geom_text(                  # Add text labels from above.
    x = 121,
    y = 4.91,
    label = "Onda Corta - Sud America",
    size = 3,                 # Override size (not loudness here).
    hjust = "left",           # Align left side of label with the point.
    vjust = "center",         # Align vertical centre of label with the point.
    angle = 30                # Rotate the text label
  ) +
  scale_x_continuous(         # Fine-tune the x axis.
    limits = c(50, 200),
    breaks = c(50, 100, 150, 200), # Specify grid lines
    minor_breaks = NULL       # Remove 'minor' grid lines.
  ) +
  scale_y_continuous(         # Fine-tune the y axis in the same way.
    limits = c(1, 9),
    breaks = c(1, 5, 9),
    minor_breaks = NULL
  ) +
  scale_colour_viridis_c() +  # Use the popular viridis colour palette.
  scale_size_continuous(      # Fine-tune the sizes of each point.
    trans = "exp",            # Use an exp transformation to emphasise loud..
    guide = "none"            # Remove the legend for size.
  ) +
  theme_light() +             # Use a simpler theme.
  labs(                       # Make the titles nice.
    x = "Tempo",
    y = "Arousal",
    colour = "Danceability"
  )
## Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
## ℹ Please use `linewidth` instead.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.

Instructions

The figure above is not great. It contains a lot of information, but does not tell a coherent story.

First, make some simple histograms and scatter plots of features that you think may give a better overview of the AI Song Contest semi-finalists. Then try to adjust the figure above to create the best visualisation you can that gives an overview of the entire dataset in a single glance.

For inspiration or help on how to make plots in R, you can refer to Hadley Wickham’s ggplot2 Book.